Methods and Apparatuses for Displaying the 3D Image of a Product

ABSTRACT

A method of displaying the 3D image of a product using an electronic device capable of capturing and displaying images comprises a code capturing step, an image procuring step, and an image displaying step. During the code capturing step, an information code on a printed media carrying the information of a product is captured. During the image procuring step, the information code is decoded for the information of the product and a 3D image of the product is retrieved from a remote location via a data transmission network. During the image displaying step, the 3D image of the product is displayed against a background, the background being a live image captured in real time or a static image retrieved from a previously saved file.

TECHNICAL FIELD

The present invention relates to methods and apparatuses of displayingthe 3D image of a product as well as a method of tracking theeffectiveness of a printed or 2D advertisement.

BACKGROUND

Advertisement is essential for the visibility of a product. Compared toother forms of advertisement such as TV commercials, printedadvertisement suffers the drawbacks of being still and limited to 2Drepresentations and thus less efficient for product promotion. Recentdevelopment in augmented reality technique allows enhancing customers'experience by supplementing and combining real world information of aproduct from a printed advertisement with computer-generated sensoryinput, such as sound, video, graphics or GPS data. For example, theLayar Vision offered by the company Layar enables the creation of layersand applications that recognize real world objects and display digitalinformation on top of them. The company Blippar offers technique thatrecognizes a product printed in media such as newspapers and magazinesand displays information regarding the product to customers. Metaiomobile SKD offered by the company Metaio provides a platform for amanufacturer or an advertisement agency to create mobile applications topresent the products to customers in a more vivid way (such as 3Dimages).

The prevailing augmented reality technique, however, mainly focuses onenlivening the presented view of a product with additional computergenerated information; it is still a one way effort made by amanufacturer or advertisement agency. The manufacturer or advertisementagency is more or less “shooting in the dark” as he does not havemeaningful feedback or input from customers. Consequently, even thoughenhanced with augmented reality technique, the products are stillpresented to a customer in a way that although the manufacturer oradvertisement agency thinks best caters customer needs but actually maynot be what the customer desires to see.

SUMMARY

It is an object of the present disclosure to enliven printedadvertisement with a method and an apparatus that provides interactionbetween customers and manufacturers and advertisement agencies so that acustomer can create a 3D view of a product against the background oftheir own choice and information of their own interest and manufacturersand advertisement agencies can collect feedback and other informationfrom customers and thus can target specific customers more precisely andpresent products to customers in a more individualized way.

According to one aspect of the present disclosure, there is provided amethod of displaying the 3D image of a product using an electronicdevice capable of capturing and displaying images. The method comprisesa code capturing step, an image procuring step, and an image displayingstep. During the code capturing step, an information code on a printedmedia carrying the information of a product is captured. During theimage procuring step, the information code is decoded for theinformation of the product and a 3D image of the product is retrievedfrom a remote location via a data transmission network. During the imagedisplaying step, the 3D image of the product is displayed against abackground, the background being a live image captured in real time or astatic image retrieved from a previously saved file.

According to another aspect of the present disclosure, there is providedan apparatus for displaying the 3D image of a product. The apparatuscomprises an image capturing unit, a processing unit, and an imagedisplaying unit. The image capturing unit is capable of capturing aninformation code on a printed media carrying the information of aproduct. The processing unit is connected to the image capturing unitand to a remote location via a data transmission network. The processingunit is capable of decoding the information code for the information ofthe product, obtaining, based on the decoded information of the product,a 3D image of the product from the remote location, obtaining, as thebackground to display the 3D image, a live image captured in real timeor a static image retrieved from a previously file saved locally or in aremote location, and processing the 3D image and the background toproduce a synthesized image displaying the 3D image against thebackground. The image displaying unit is connected to the processingunit, and is capable of receiving the synthesized image from theprocessing unit and displaying the synthesized image.

According to a further aspect of the present disclosure, there isprovided a method for tracking the effectiveness of a printedadvertisement that can be captured and identified by an electronicdevice capable of displaying a 3D image. The method comprises a codecapturing step, an image procuring step, an image displaying step, and adisplay data collection step. During the code capturing step, aninformation code on a printed media carrying the information of aproduct is captured with the electronic device. During the imageprocuring step, the information code is decoded for the information ofthe product and a 3D image of the product is retrieved from a remotelocation. During the image displaying step, the 3D image of the productis displayed on the electronic device against a background, thebackground being a live image or a static image from a previously savedfile; and a display data collection step, in which display data of the3D image is recorded and saved.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate several embodiments of theinvention and together with the description, serve to explain theprinciples of the invention.

FIG. 1 shows a method of displaying the 3D image of a product using anelectronic device capable of capturing and displaying images;

FIG. 2 is a flow chart illustrating the process of displaying the 3Dimage of a product;

FIG. 3 shows a method of displaying the 3D image of a product using anelectronic device capable of capturing and displaying images accordingto the second embodiment of the present disclosure;

FIG. 4 illustrates an apparatus for displaying the 3D image of a productaccording to the third embodiment of the present disclosure;

FIG. 5 illustrates a method for tracking the effectiveness of a printedadvertisement that can be captured and identified by an electronicdevice capable of displaying a 3D image.

DETAILED DESCRIPTION

Reference will now be made in detail to the present exemplaryembodiments, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts. While thedescription includes exemplary embodiments, other embodiments arepossible, and changes may be made to the embodiments described withoutdeparting from the spirit and scope of the invention. The followingdetailed description does not limit the invention. Instead, the scope ofthe invention is defined by the appended claims and their equivalents.

FIG. 1 shows a method 100 of displaying the 3D image of a product usingan electronic device capable of capturing and displaying images. Theelectronic device can be a tablet computer such as Android or iPad, or amobile device such as the Smart Phone with a built-in or attachedcamera. The products to be displayed in 3D images include a wide rangeof consumer goods, such as cars, household articles like refrigeratorsand washing machines, clothes, books, and laptop computers.

As shown in FIG. 1, the method 100 may comprise a code capturing step101, an image procuring step 102, and an image displaying step 103.

At the code capturing step 101, an information code on a printed mediacarrying the information of a product is captured with the electronicdevice. The information code could be of any form that carries theinformation regarding a product, such as a bar code, a 2D code, a 3Dcode, or just an image of the product. The information code may beprinted on a newspaper, a magazine, or a product pamphlet; it may alsobe painted outdoors on a newspaper kiosk, a cab, a highway bulletin, ora subway station. The information code may also be shown on a computerscreen, a TV screen, or screens of any other electronics.

At the image procuring step 102, the information code captured in thecode capturing step 101 is decoded for the information of the product.This can be done with well-known deciphering technology. For example,for a bar code, the decoding can be done with the barcode readingtechnology. As another example, a 2D code may be decoded by a computerprogram, such as an iPhone app, through matching the 2D code with a codein a local or remote database server. The information may include thename, serial number, manufacturer of a product, or any otherinformation.

After the decoding, a 3D image of the product is retrieved from a remotelocation via a data transmission network. The 3D image may be 3Dcomputer graphics that use a three-dimensional representation ofgeometric data (often Cartesian) that is stored in a storage device forthe purposes of performing calculations and rendering 2D images; inother word, a 3D image herein means graphic data that can be calculatedand rendered in a 2D screen from a plurality of perspectives. Herein,the 3D images are stored in a remote location, such as a databaseaccessible to the public, often provided and maintained by anadvertisement agency or a product distributor. A customer can access thelocation to retrieve the 3D images, e.g., using an iPhone app, via adata transmission network, which may be the internet, a LAN, or anynetwork that is capable of transmitting data.

At the image displaying step 103, the 3D image of the product isdisplayed on the electronic device against a background. The backgroundmay be a live image captured by the electronic device in real time. Thebackground can also be a static image retrieved from a previously savedfile, which is stored in the image displaying device or stored in aremote location and retrieved via a data transmission network (internet,LAN, or any network capable of transmitting data). The background may beany image suitable for the product whose 3D image is to be displayed.For example, for a car, the background may be the garage of a customeror even the customer himself; while for a refrigerator or a bed, thebackground can be the interior of the kitchen or the bedroom of a house.Displaying the 3D image of a product against a background can deliver amore vivid representation of the product to a customer.

The 3D image may initially be displayed in one view angle; it may alsobe displayed from continuously varying view angles (i.e., beingrotated). In addition, a customer can actively change the way ofdisplaying of the 3D image, such as the view angle and the position ofthe 3D image, which will be described later with references to FIG. 2.

The method may further comprise a background capturing step 104 forcapturing an environment in real time as the static image and saving thestatic image in a local or remote location. The background capturingstep 104 can be performed at any time, for example, after the codecapturing step 101, after the image procuring step 102, or after theimage displaying step 103. As an example, in FIG. 1, the step 104 isperformed after the image displaying step 103.

In addition to displaying the 3D image of the product and thebackground, relevant information of the product, such as the price,evaluation of other customers, the product's website, description of theparts of the product, and the manufacturer, can be displayed alongsidewith the 3D image or the background in a form such as a floating tag(floating with the 3D image of the product), a bar, or even soundvoicing such information.

FIG. 2 is a flow chart illustrating the process of displaying a 3D imageof a product.

At step 1031, the size of the 3D image is calculated based on thephysical distance of the information code from the electronic device,which can be done by means of, for example, a camera matrix known to oneskilled in the art.

At step 1032, an image is retrieved as the background for displaying the3D image. As stated above, the background may be a live image capturedby the electronic device in real time. The background can also be astatic image retrieved from a previously saved file, which is stored inthe electronic device or stored in a remote location and retrieved via adata transmission network (internet, LAN, or any network that is capableof transmitting data).

At step 1033, the background is displayed on the image displayingdevice.

At step 1034, the position of the 3D image of the product is determinedbased on the position of the information code captured by the electronicdevice with respect to the electronic device. For example, the positionof the 3D image, usually represented by a vector, can be obtained bytransforming the vector representing the position of the informationcode with a predetermined transform matrix.

Alternatively, other ways may be employed to determine the position ofthe 3D image with respect to the background. For example, it can bedetermined based on a user-specified position. The user can specify theposition of the 3D image of the product by touching the backgrounddisplayed on the display of the electronic device so as to choose wherethe product should be located.

At step 1035, the 3D image is displayed, with the size calculated asabove and the position determined as above, against the background.

At step 1036, a user action is sensed by the electronic device. The kindof user action to be sense can be selected according to needs. Forexample, the change in the attitude of the device can be sensed so as tochange the view angle of the 3D image of the product according to thechange in the attitude of the electronic device; the change in thephysical position of the information code captured by the device can besensed so as to change the position of the 3D image of the product withrespect to the background; the distance between the information code andthe device may be sensed to as to change the size of the 3D image of theproduct according to the distance.

A decision 1037 determines whether the attitude of device is changed.When the decision 1037 determines that the attitude of the device isnoticeably changed, the view angle of the 3D image is changedaccordingly at step 1038, and the process goes back to the step 1035 torefresh the displayed 3D image of the product.

When the decision 1037 determines that the attitude of the device is notnoticeably changed, a decision 1039 is made to determine whether thephysical position of the information code captured by the device ischanged (i.e., whether the use move the electronic device with respectto the information code). When the decision 1039 determines thatposition of the information code is changed, the position of the 3Dimage is changed accordingly at step 1040, and the process goes back tothe step 1035 to refresh the displayed 3D image of the product.

The process illustrated in FIG. 2 initially shows the 3D image of theproduct with its size and position determined by the physical distanceand position of the information code or specified by the customer. Thesize and position of the 3D image, as well as other features of the 3Dimage and features of the background, can instead be determined byartificial intelligence or intelligent judgment.

In the process illustrated in FIG. 2, the view angle and position of the3D image are changed according to the user action sensed at the step1036. Instead of that, the step 1036 can be substituted with a step ofcomputer vision analysis based on a series of images of the codeinformation code successively captured by the electronic device in thecode capturing step 101, wherein the series of images of the informationcode are analyzed by means of computer vision to deduce whether and howthe position, view angle, and distance of the information code withrespect to the electronic device are changed; and depending on thededuced change of the position, view angle, and distance of theinformation code with respect to the electronic device, the position ofthe 3D image with respect to the background, the view angle of the 3Dimage of the product, and the size of the 3D image are varied. Computervision, a field that includes methods for processing, analyzing, andunderstanding images, is known art and thus its detailed description isomitted.

The process illustrated in FIG. 2 only shows changing the way ofdisplaying of the 3D image of a product in two aspects, i.e., changingthe view angle and the position of the 3D image. However, one skilled inthe art knows that the way for displaying the 3D image can be varied inmany aspects. For example, the size of the 3D image can be changedaccording to the distance between the information code and theelectronic device; the position of the 3D image of the product withrespect to background can be changed according to the movement of thefinger of a customer on the touch screen of the electronic device; the3D image of the product can be zoomed in or zoomed out according to therelative movement of two fingers of the customer on the touch screen ofthe electronic device or according to the double clicking by thecustomer.

Moreover, the process can change the way the background is displayedjust like changing the way the 3D image of the product as describedabove. For example, if the electronic device senses a user action forzooming in or zooming out the background (e.g., the movement of twofingers of the customer on the background image), the process will goback to the step 1033 and refresh the background image. Or the processcan replace the background image being displayed with another one, inwhich case, the process goes back to the step 1032 and retrieve anotherbackground image.

In addition to displaying the 3D image of a product, the background, andthe relevant information of the product, the screen of the electronicdevice can also display a GUI widget such as a toolbar, on whichon-screen buttons, icons, menus, or other input or output elements areplaced so that a customer can take pictures, share the images, purchasethe product or pursue other activities by clicking the correspondingon-screen buttons, icons, menus, or other input or output element on theGUI widget.

The process described above comprises steps 1031 to 1040. To one skilledin the art, one or more steps can be saved or substituted with others.For example, step 1031 can be omitted, with the 3D image displayed at apredetermined size; or the step 1031 can be substituted with a step thatdetermines the size of the 3D image according to the input of thecustomer. Furthermore, one skilled in the art can add more stepsaccording to practical needs. For example, before the step 1032 thefollowing step can be inserted: selecting a number of suitablebackground images based on the nature of the product, showing the iconsof the selected background images for the customer to select the one helikes most.

In addition, sequences of different steps as illustrated in FIG. 2 canbe varied; in other words, the steps 1031-1040 can be performed in anorder different from that shown in FIG. 2. For example, the decision1039 can be performed prior to the decision 1037. For another example,the step 1031 can be performed after the step 1032 so that the nature ofthe background image can be taken into account during calculation of thesize of the 3D image of the product; or it can be performed after thestep 1033.

Also, one skilled in the art understands that some steps as illustratedin FIG. 2 can be combined into one. For example, the step 1031 and thestep 1034 can be combined into one, which, for example, determines thesize and the position of the 3D image with one predetermined cameramatrix based on the distance and position of the information code.

Moreover, the above mentioned steps 1031-1040 can be implemented asevents. In other word, instead of being structured one by one as in FIG.2, the steps may be events to be triggered and handled during displayingof the 3D images.

FIG. 3 shows a method 300 of displaying the 3D image of a product withan electronic device capable of capturing and displaying imagesaccording to the second embodiment of the present disclosure. The methodaccording to the second embodiment also comprises the code capturingstep 301, the image procuring step 302, and the image displaying step303; the steps 301, 302, and 303 function similarly as the steps 101,102, and 103 shown in FIG. 1 as described above and thus their detaileddescription will not be repeated. In addition to the steps 301, 302, and303, the method 300 further comprises a first information monitoringstep 3051 after step 301, a second information monitoring step 3052after step 302, and a third information monitoring step 3053 after step303. The first, second, and third information monitoring steps 3051,3052, and 3053 monitor, respectively, the data obtained and generated inthe code capturing step 301, the image procuring step 302, and the imagedisplaying step 303; and may transmit the data obtained and generated toa server for further processing.

The first information monitor step 3051 obtains and gathers theinformation generated during capturing of an information code on aprinted media. In one embodiment, the electronic device analyzes whichmedium (i.e., newspaper, magazine, product pamphlet, kiosk, cab, highwaybulletin, or subway) the information code is located. In anotherembodiment, the electronic device will transmit the information to aremote server, and the remote server has a computer program that thatcompiles the data transmitted from different customers so as to extractknowledge such as which medium is most visible to customers.

In the second information monitoring step 3052, the data obtained andgenerated includes the information obtained through decoding theinformation code and the number of times the information code of theproduct is captured and decoded. In one embodiment, the computer programon the electronic device counts and records each time that it capturesand decodes the information code. In another embodiment, the computerprogram will transmit the information to the remote server, and thecomputer program of the server counts and records each time theinformation code of the product is captured and decoded. In addition,since a user must log into the computer program before it displays 3Dimages, the specific user information may also be recorded, along withthe number of times the information code of the product is captured anddecoded.

In the third information monitoring step 3053, the data obtained andgenerated includes the start and end time the 3D image of the product isdisplayed. In one embodiment, the computer program on the electronicdevice records the start and end time the 3D image of the product thatthe device displays. In another embodiment, the computer program on theelectronic device transmits the start and end time the 3D image of theproduct that it displays to a remote server.

The above data can be used to analyze the purchasing behaviors ofcustomers or the efficiency of the advertisement. Specifically, in atraditional printed media, an advertiser has no idea regarding how manytimes the advertisement has been viewed by customers and how long dideach customer view such advertisement. Consistent with the embodimentsof the present invention, an advertiser knows who viewed theadvertisement in a 3D image, how many times the advertisement has beenreviewed, how long a specific customer viewed the advertisement in a 3Dimage, and which medium is the most visible to customers. Based on thesedata, an advertiser may modify and improve its advertising campaign suchas selecting the right advertising medium and targeting those customerswho have viewed the advertisement for a certain number of times and/orfor a certain duration for follow-up.

The above description shows using the monitoring steps for data foranalyzing the purchasing behaviors of customers or the efficiency of theadvertisement. The steps can also be used for individualizing the 3Dimages and background delivered to the customers. For example, based onthe information about the furniture one customer ever viewed, the remoteserver can analyze the customer's propensity regarding certainfurniture, such as the color and the style, and in the future, the imageprocuring step can retrieve the proper 3D image of the furniture thatbest suit the customer's propensity. For another example, whenpurchasing furniture, a customer usually captures a picture of one roomof his house as the background. The computer program on the electronicdevice may store the room pictures either locally or remotely so thatnext time the customer views a piece of similar furniture, the imagedisplaying step can display the 3D image of the piece of furnitureagainst the right room picture.

The method 300 as described above comprises three information monitoringsteps 3051-3053. According to practical needs, one or two informationmonitoring steps can be saved or more information monitoring steps canbe added.

Furthermore, instead of being implemented sequentially as shown in FIG.4, the information monitoring steps 3051-3053 can be implemented asevents that are triggered by, respectively, the actions of capturing aninformation code, decoding the information code, retrieving the 3D imageof a product, and displaying the 3D information.

FIG. 4 illustrates an apparatus for displaying the 3D image of a productaccording to the third embodiment of the present invention.

As shown in the FIG. 4, the apparatus 400 comprises an image capturingunit 401, a processing unit 402, an information monitoring unit 403, andan image displaying unit 404.

The image capturing unit 401 is capable of capturing an information codeon a printed media carrying the information of a product. Theinformation code could be of any form that carries the informationregarding a product, such as a bar code, a 2D code, a 3D code, or justan image of a product. The information code may be printed on anewspaper, a magazine, or a product pamphlet; it may also be paintedoutdoors on a newspaper kiosk, a cab, a highway bulletin, or a subwaystation. The information code may also be shown on a computer screen, aTV screen, or the screens of any other electronics.

In addition, the image capturing unit 401 can capture an image in realtime as the static image and send the image to the processing unit 402to be described below, so that the static image can be saved in a localor remote location for future use.

The processing unit 402 is connected to the image capturing unit 401 andto a remote location 406 via a data transmission network. The processingunit 402 is capable of decoding the information code for the informationof the product. It can also obtain, based on the decoded information ofthe product, a 3D image of the product from the remote location.Moreover, it can obtain, as the background to display the 3D image, alive image captured in real time or a static image retrieved from apreviously file saved locally or in a remote location. The processingunit 402 can further process the 3D image and the background to producea synthesized image displaying the 3D image against the background. Inthe synthesized image, relevant information of the product, such as theprice, evaluation of other customers, the product's website, descriptionof the parts of the product, and the manufacturer, can also be providedalongside with the 3D image or the background in a form such as afloating tag (floating with the 3D image of the product), a bar, or evensound voicing such information.

The processing unit 402 is also capable of determining the initial sizeand position of the 3D image according to the physical distance andposition of the information code or specified by the customer. It canalso determine the size and position of the 3D image, as well as otherfeatures of the 3D image and features of the background, throughartificial intelligence or intelligent judgment.

The apparatus may further comprise a sensor 405 connected to theprocessing unit 402 for sensing user actions, so that the processingunit can receive user actions from the sensor and perform correspondingactivities based on the user actions, for example, varying the viewangle and the size of the 3D image of the product depending on thechange of the attitude of the apparatus by the user. The sensor 405 canbe of any form for sensing any user action. For example, it can be atouch screen for sensing the touching and movement of fingers or anyother objects; or it can be a microphone for receiving the user's oralcommands.

In fact, based on the information received from the sensor, theprocessing unit 402 can perform a variety of operations on the 3D imageof the product and the background, among which are: calculating the sizeof the 3D image of the product based on the physical location of theinformation code, and processing the 3D image into the calculated size;determining the position of the 3D image with respect to the backgroundon a user-specified position, and thus having the 3D image of theproduct displayed at the determined position in the synthesized image;changing the size of the 3D image according to the distance between theinformation code and the electronic device; changing the position of the3D image of the product with respect to background according to themovement of the finger of a customer on the touch screen of theelectronic device; zooming in or zooming out the 3D image of the productor the background image according to the relative movement of twofingers of the customer on the touch screen of the electronic device oraccording to the double clicking by the customer; and zooming in orzooming out the background image according to the movement of twofingers of the customer on the background image.

Moreover, the image capturing unit 401 is capable of successivelycapturing a series of images of the information code. And the processingunit is capable of analyzing the series of images of the informationcode by means of computer vision to deduce whether and how the position,view angle, and distance of the information code with respect to theelectronic device are changed. And the image displaying unit is capableof, depending on the deduced change of the position, view angle, anddistance of the information code with respect to the electronic device,varying the position of the 3D image with respect to the background, theview angle of the 3D image, and the size of the 3D image.

In addition, the processing unit can have the image displaying unit 404display a GUI widget such as a toolbar, on which on-screen buttons,icons, menus, or other input or output elements are placed so that acustomer can take pictures, share the images, or pursue other activitiesby clicking the corresponding on-screen buttons, icons, menus, or otherinput or output element on the GUI widget.

The image displaying unit 404 is connected to the processing unit 403.It receives the synthesized image from the processing unit and displaysthe image.

The information monitoring unit 403 is connected to the image capturingunit 401, the processing unit 402, and the image displaying unit 404 andis also connected to a remote server 407. The information monitoringunit 403 is capable of monitoring the data obtained and generated by theimage capturing unit, the processing unit, and the image displayingunit, and is capable of transmitting the data obtained and generated tothe remote server 407 for data processing. The remote server 407 may beprovided and maintained by an advertisement agency or a productdistributor; it may be located at the same place as the remote location406.

The data obtained and generated by the information monitoring unit 403includes the information obtained through decoding the information codeand the number of times the information code of the product is capturedand decoded. The data obtained and generated by the informationmonitoring unit 404 can also include the start and end time the 3D imageof the product is displayed.

The processing unit 402 or the information monitoring unit 404 asdescribed above can be a chip, such as an IC or a microprocessor, whichcomprises at least circuitry, memory, and processor for storing data andfor performing the functions as described above for the units 402 and404. The functions of the unit 402 or 404 as described above can berealized through programs stored and run in the chip.

In the above description, the processing unit 402 and the informationmonitoring unit 404 are described as separate parts; they can also beintegrated as one unit. In addition, as shown in FIG. 4, the units 402and 404 can be a part of a CPU 408; or the processing unit 402 and theinformation monitoring unit 404 can be virtual modules implemented inthe CPU 408.

The apparatus described above can be a mobile device with a built-in orattached camera. For example, the apparatus can be a tablet computersuch as Android or iPad, or a mobile phone such as the Smart Phones, inwhich case the processing unit 402 and the information monitoring unit403 can be the virtual modules of the CPU of the tablet computer or themobile phone.

FIG. 5 illustrates a method 500 for tracking the effectiveness of aprinted advertisement that can be captured and identified by anelectronic device capable of displaying a 3D image. The printedadvertisement contains a bar code, a 2D code, or a 3D code carrying theinformation of the product.

As shown in FIG. 5, the method 500 comprises a code capturing step 501,an image procuring step 502, an image displaying step 503, and a displaydata collection step 506. The steps 501, 502, and 503 function similarlyas the steps 101, 102, and 103 of the method 100 described above andthus their detailed description is omitted.

The display data collection step 506 comprises: transmitting or savingthe data to a local or remote location; analyzing the data for theeffectiveness of the printed advertisement. In one embodiment, thecomputer program on the electronic device counts and records each timethat it captures and decodes the information code. In anotherembodiment, the computer program will transmit the information to aremote server, and the remote server has a program that counts andrecords each time the information code of the product is captured anddecoded. In addition, since a user must log into the computer programbefore it displays 3D images, the specific user information may also berecorded, along with the number of times the information code of theproduct is captured and decoded.

The display data collection step 506 comprises obtaining the displaydata including the start and end time the 3D images of the products aredisplayed. In one embodiment, the computer program on the electronicdevice records the start and end time the 3D image of the product thatit displays. In another embodiment, the computer program on theelectronic device transmits the start and end time the 3D image of theproduct that it displays to a remote server.

The above data can be used to analyze the purchasing behaviors ofcustomers or the efficiency of the advertisement. Specifically, in atraditional printed media, an advertiser has no idea regarding how manytimes the advertisement has been viewed by customers and how long dideach customer view such advertisement. Consistent with the embodimentsof the present invention, an advertiser knows who viewed theadvertisement in a 3D image, how many times the advertisement has beenreviewed, how long a specific customer viewed the advertisement in a 3Dimage, and which medium is the most visible to customer. Based on thesedata, an advertiser knows whether a particular advertisement campaign issuccessful or not and how it compared to other advertisement campaigns.The advertiser also knows which customers are more interested in theadvertised product based on the number of times these customers haveviewed the advertisement and the duration that each of these customersreviewed the advertisement. As a result, the advertiser may modify andimprove its advertising campaign such as selecting the right advertisingmedium and targeting those customers who have viewed the advertisementfor a certain number of times and/or for a certain duration forfollow-up.

In the preceding specification, various preferred embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various other modifications and changes may be madethereto, and additional embodiments may be implemented, withoutdeparting from the broader scope of the invention as set forth in theclaims that follow. The specification and drawings are accordingly to beregarded in an illustrative rather than restrictive sense.

For example, the above embodiments generally describe displaying 3Dimages based on a 2D code on a printed advertisement. One skilled in theart would understand that the invention also discloses displaying 3Dimages based on a 2D code shown on a computer screen instead of aprinted advertisement. One skilled in the art would also understand thatthe invention also discloses displaying 3D images based on a 2D code ofan object, even though the object is not displayed for advertisement.

Other embodiments of the invention will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with the true scope and spiritof the invention being indicated by the following claims.

We claim:
 1. A method of displaying the 3D image of a product using anelectronic device capable of capturing and displaying images,comprising: a code capturing step, in which an information code on aprinted media carrying the information of a product is captured; animage procuring step, in which the information code is decoded for theinformation of the product and a 3D image of the product is retrievedfrom a remote location via a data transmission network; an imagedisplaying step, in which the 3D image of the product is displayedagainst a background, the background being a live image captured in realtime or a static image retrieved from a previously saved file.
 2. Themethod according to claim 1, wherein in the image displaying step, thesize of the 3D image of the product is calculated based on the physicaldistance of the information code from the electronic device, and the 3Dimage of the product is displayed with the calculated size.
 3. Themethod according to claim 1, wherein in the image displaying step, theposition of the 3D image with respect to the background is determinedbased on a user-specified position, and the 3D image of the product isdisplayed at the determined position.
 4. The method according to claim1, wherein a series of images of the information code information codeare successively captured by the electronic device, and in the imagedisplaying step, the series of images of the information code areanalyzed by means of computer vision to deduce whether and how theposition, view angle, and distance of the information code with respectto the electronic device are changed, and depending on the deducedchange of the position, view angle, and distance of the information codewith respect to the electronic device, the position of the 3D image withrespect to the background, the view angle of the 3D image, and the sizeof the 3D image are varied.
 5. The method according to claim 1, whereinthe device is capable of sensing a user action; in the image displayingstep, a user action is sensed; and in the image displaying step,depending on the sensed user action, at least one of the position of the3D image with respect to the background, the view angle of the 3D imageof the product, and the size of the 3D image of the product is varied.6. The method according to claim 1, further comprising a backgroundcapturing step, in which a picture is captured in real time as thestatic image and saved in a local or remote location.
 7. The methodaccording to claim 1, further comprising at least one of the followingsteps: a first information monitoring step after the code capturingstep, a second information monitoring step after the image procuringstep, and a third information monitoring step after the image displayingstep, wherein the first, second, and third information monitoring stepsmonitor, respectively, the data obtained and generated in the codecapturing step, the image procuring step, and the image displaying stepand transmit the data obtained and generated to a remote server forprocessing.
 8. The method according to claim 7, wherein in the secondinformation monitoring step, the data obtained and generated includesthe information obtained through decoding the information code and thenumber of times the information code of the product is captured anddecoded.
 9. The method according to claim 7, wherein in the thirdinformation monitoring step, the data obtained and generated includesthe start and end time the 3D image of the product is displayed.
 10. Themethod according to claim 7, wherein the information monitored andtransmitted by the first, second, and third information monitoring stepsare used to analyze the propensity of a customer toward the product sothat in the image procuring step, a 3D image of the product that suitsthe propensity is retrieved, and in the image displaying step, the 3Dimage is displayed against a background that suits the propensity. 11.The method according to claim 1, wherein the electronic device is amobile device with a built-in or attached camera.
 12. The methodaccording to claim 1, wherein the information code is a bar code, a 2Dcode, or a 3D code.
 13. An apparatus for displaying the 3D image of aproduct, comprising an image capturing unit, a processing unit, and animage displaying unit, wherein the image capturing unit is capable ofcapturing an information code on a printed media carrying theinformation of a product; the processing unit is connected to the imagecapturing unit and to a remote location via a data transmission network,the processing unit is capable of decoding the information code for theinformation of the product, obtaining, based on the decoded informationof the product, a 3D image of the product from the remote location,obtaining, as the background to display the 3D image, a live imagecaptured in real time or a static image retrieved from a previously filesaved locally or in the remote location, processing the 3D image and thebackground to produce a synthesized image displaying the 3D imageagainst the background; and the image displaying unit is connected tothe processing unit, and is capable of receiving the synthesized imagefrom the processing unit and displaying the synthesized image.
 14. Theapparatus according to claim 13, wherein the processing unit is capableof calculating the size of the 3D image of the product based on thephysical distance of the information code from the electronic device,and processing the 3D image into the calculated size.
 15. The apparatusaccording to claim 13, wherein the processing unit is capable ofdetermining the position of the 3D image with respect to the backgroundon a user-specified position, and thus having the 3D image of theproduct displayed at the determined position in the synthesized image.16. The apparatus according to claim 13, wherein the image capturingunit is capable of successively capturing a series of images of theinformation code, the processing unit is capable of analyzing the seriesof images of the information code by means of computer vision to deducewhether and how the position, view angle, and distance of theinformation code with respect to the electronic device are changed, andthe image displaying unit is capable of, depending on the deduced changeof the position, view angle, and distance of the information code withrespect to the electronic device, varying the position of the 3D imagewith respect to the background, the view angle of the 3D image, and thesize of the 3D image.
 17. The apparatus according to claim 13, furthercomprising a sensor connected to the processing unit for sensing a useraction, wherein the processing unit is capable of receiving theinformation regarding the user action from the sensor and is capable of,depending on the sensed user action, varying the view angle, the size ofthe 3D image of the product, or the position of the 3D image withrespect to the background in the synthesized image.
 18. The apparatusaccording to claim 13, the image capturing unit is capable of capturingan image in real time as the static image and saving the static image ina local or remote location.
 19. The apparatus according to claim 13,further comprising a information monitoring unit connected to the imagecapturing unit, the processing unit, and the image displaying unit andconnected to a remote server, wherein the information monitoring unit iscapable of monitoring the data obtained and generated by the imagecapturing unit, the processing unit, and the image displaying unit, andis capable of transmitting the data obtained and generated to the remoteserver for processing.
 20. The apparatus according to claim 19, whereinthe data obtained and generated includes the information obtainedthrough decoding the information code and the number of times theinformation code of the product is captured and decoded.
 21. Theapparatus according to claim 19, wherein the data obtained and generatedincludes the start and end time the 3D image of the product isdisplayed.
 22. The apparatus according to claim 13, wherein apparatus isa mobile device with a built-in or attached camera.
 23. The apparatusaccording to claim 13, wherein the information code is a bar code, a 2Dcode, or a 3D code.
 24. A method for tracking the effectiveness of aprinted advertisement that can be captured and identified by anelectronic device capable of displaying a 3D image comprising: a codecapturing step, in which an information code on a printed media carryingthe information of a product is captured with the electronic device; animage procuring step, in which the information code is decoded for theinformation of the product and a 3D image of the product is retrievedfrom a remote location via a data transmission network; an imagedisplaying step, in which the 3D image of the product is displayed onthe electronic device against a background, the background being a liveimage or a static image from a previously saved file; and a display datacollection step, in which display data of the 3D image is recorded andsaved.
 25. The method according to claim 24, wherein the display datacollection step comprises: transmitting the data to a remote server;analyzing the data for the effectiveness of the printed advertisement.26. The method according to claim 24, wherein the printed advertisementcontains a bar code, a 2D code, or a 3D code carrying the information ofthe product.
 27. The method according to claim 24, wherein the displaydata collection step comprises obtaining the display data including thestart and end time the 3D images of the products are displayed.